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• INTRODUCTION 

In a year of rigorous re-exaini?tation of schhol budgets, 
it proved a good choice to devote this, 28th Rutgfers Guidance 
^Conference to Accountability, To be sure, the term has become 
a catchword, and that process ,of ten foreshadows dissipation of 
the concept.^ Jn this case, it is predictable that continuing 
pressure to define accountability in education will continue, 
Th^ thrust in that direction may well be stronger for counselors 
than \ for .ether educators. If so, then counselors should be in 
the fxjrefront. This Conference served to answer few questions; 
scSme old questipns were refined, perhaps some new pnes were i 
raised. At most, it represe.its a step in an important direction 
Hopefully, it has, for som.'2, helped *to define some "next steps." 
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iree days before the scheduled date of the 28th, Rutgers 
Guidance Conference; James £• Allen, Jr., the keynote speaker, 
was 'killed in an airplane crash. This volume is dedicated to 
his meijiory. Through a rare combination of h\iman attributes — 
political coujrage and personal integrity — he was an inspiratiorT 
to many. 



V. 



— Editor 



L . ■ - . . • ■ 

A' particular word of thanks' goes to Dr* Edward J. Meade, Jr. 
of the Forid Foundation. On unreasonably short notice he agreed * 
to read^DrV Allen's pa^er. In the fallowing pages. Dr. |Mlen's / 
paper is presented in full, with D^. Mead's comments ir)J 
parentheses. , ' . . \ , 



I. 



I 



r 



ERIC 



f 

• < * 



ACCOUNTABILITY WHOSE ' RESI^DNSIBILITY? 



James E. Alleii., Jr. 

This is arT opportunity for me, but with. all' due respect to 
the Association and University^ it's Iqss of an opportunity than 
it ^s a chance for me to repay a debt that I can^ never repa^y/ Jim 
Allen and I >;ere associate!s for some 12 years, guess I got 

to know him best when I had a fortuitous episode >anNiny life in 
1969. For s^e reason, I as a p^cticing democrat was called tp 
Washington by a practicing republican the name of Robert Finch* 
"Come down and jgive me two days so we can calk about issues in 
Education, said Robert Finch* In the coursevof those two days 
(which by the way took two weeks) the present administration saw 
tit to appoint what in my inind was one of the best, if not t^e 
best, pe^80h;in education,, who understands the seiisitive role be- 
twefen state, and federal governments in education, Jcua6s Allen. 
Jim was talked into the job* He earlier turned it down when 
{offered the job by President Kennedy, as some of you may know*- He 
had a condition, that is, he couldn't come until May*" So I hung 
around from January 69 to May or June of 69 until Jim Allen got- on 
board. ^ ,It's iconic because I find myself sitting in for my now 
present boss, who was commissioner of education under Lyndon 
Johnson, waiting for Jim Allen* I gu^ss L'll always wait for Jim 
Allen* He was a remarksUble individual* Dean Solvwebol, I think > 
caught the ^pirit of Jim Allen, his tougljness and firmness witti 
riespect to moral issuers in oiir business, Jrii segregation, in Cam- 
bodia* He was accounftable; you can see' that in his record through- 
out Jiis career* He did ihdeed put 'his jobs^on the iine*. It was 
a death wish in some way s •/V We all knew wttat- would happen as he 
spoke out as he did* But hk at least wanted to be able tq face 
himself and his constituentsV^ith a clebr conscience, and be did^ 
So I Qome to you with mixed fteeling« frankly, on othe;: transifi- 
c^ion, I'd say, ••gee its a great opportunity for/me^to talk t<5- ^ 
people fyom my home state**, 'Ibut" I come more in payment of a deb^ 
that will never be repayed* I guess* * I'm going to speak or 'read 
rather Jim's text, which he prepared earlier* No apologias for 
the text, it's what Jim wanted to say, but plenty pf apologies , 
fpr h()w it will be said this morning* Jim title6 his^ .remarks • 



/••Accountability — Whose Responsibility? •• 

^ . . ■ -y. ■ , ^ 

Welcome to another stop on the .accountability bandy^agonl 



. . .What- 1 want to know is are we here, just as barker? at the 
latest medicine show or are, we here a? real 'physicians, Beeking[ 
amiew cure? ' -kx ^ 



^ A^e wd just making Mig talk ^o sell the same old 'elixir 
that will soothe and ccnaSort bUt won't really help'or change ,^ / 
or are we seriously brewii^g a new wonder drug? ^ 



Certriniy, thS Ills and problems of education today , Require 
something more than a mere soothing syrup, and the real question^ 
is: -Are we just trying to make -the- schools look goc>d;>f>r really, 
be good? (I'll pauS'e here because*^ Jim's . words just haiJnt me.) 

,THe answer to this question lies, it seems to m'e in another 
question: Why accouatability now? (Why are we here ,in 1971, 
worrying about aocountabilityj?^ I heard\earlier today that this - 
has been* planned for this conference foif something lik^'two years.) 

As a matter of logic/ accountability should 'always h^ve been - 
a major factqr in education — and of^CQurs^,^ it -has not been ig- 
no;:ed— but too often.it has peen hit br mis^*^in operation an4 
woefully unscientifit in degign^.and application. . (Now, we all J 
have gpne through the courses, .-have we* nop where we #ay we do 
irt on behalf of the children,^ and^ we d^esign bur programs andj needs 
afnd so forth ,^ but by and lar^' its heSn rhetoric rather than 
action) . ' /• • ^ . ' ^ 

Is itr? present popularity a response^^tb outside criticism 
and laggi'ng support, x)r is^it a response -to an inner s^nsci. of ^ 
failure? If it is j^ist the former, tthat^ w6 arfe now reacting to: 
sometjiing we ^ot from the outside,) we aire wasl:ing out. time and 
might^^ jwell go home and forget the who 1^ thing, ^ for .^uch a 
basisi^^r accountability can all too easily di-stort its purpose- 
and limit it3 -usefulness. * ^ 

/•* ' ' ' . ^' \' ' ' \ 

^ Certainly , our school systems want-to win back losr confidence; 
^nd gain moTC-. support, and accountability is the, best and surest 
way to do this. But sucli a result .w^fl be attained only if ac- 
countJ&bility has fis its aim genuine improvement an^i the elimihaticm 
of deficiencies that ican stand as jifstif ipation of confidence 

and increased- support : ) 

• • . . ^ • .... -t . • 

Unfo|:tuna,t^ly,\in the minds of >oo many, -accountability haaf 
-much too' much of a public, relations, orientation, and this could/- 
be f atal^-destroying the potential of one of the most hopeful ^ ^ 
deyelopments. in education,! and making-iiboust another one of . > 
those cycj^icfi^^l -fads th'at sfeem to sweep*-tOTOugh tlie educational 
world causing great commotion but leaving.Vhings little changed 
♦a£ter their passing. ^ . . ' ^ 

1/ Accountability ly more often f*an not con^^idered in ^he na^rrow 
sens6*pf assesfsgibnt and measur^enl rather than in its -.broader 
and more definitive' meaning of being responsible and JLiable. 
Thus '^defiried, the question becomes: ^responsible to whom?' 

I^N^e accept' the premise that the school .^belong to the"; ^ 
people (atad certainly J^m Allen believed that) — the premise* 
upon, whicn our public school system has been developed^ — (and« - 
that is the premise) then the answer is obrfiously (we are : 
accoxyitabie to) the public. But what is the publifc? 

' . : Is it society as a whole? . It has geiV^rally* been acceptied 
that the -education 'system serves society by 'producing ^valuable 



members/ and this seifyice encompasses* e^ferything froHV- training' 
people to be good- citizens, tb teaching' them^' useful skills, to ' ' 
making it passible or eact\* individual to reaqh self-fulf illment, 

;^ Is it tshe taxpayers? (Not in the l^imit^^ense of the ^ 
•taxpayers association^ that all -of us have met from time to time,); 
Here the responsibility » seems very. clear. 'Th^ taxpayer . supports 
the schools and deserves the assurance that his inv(&stment is • 
^worthwhile. The concern here generally is that the- Schools be 
run effdcimtly and that ^^he ijax. bill be not too great. . ./> 

Is It the parents? They look to the schools to educate . / 
their children*, and by this -they usually mean that, they . fexpect 
their children to be able to participate ef fectiyeiy in, society 
and to be prepar.ed to make a living.^ guess I VouldJ^kve to. 
say, aside from Jim's remarks, too. many parents think itfis' a 
place .to 'keep the kids quiet.)* v •'^ ^" .J.'^ ^: 

it thQ students? This is the most ^iirect and the, most 
difficult respdnsibilitjf , for the school musl: deal with the dual 
task- of considering andif satisfying the needs and interests of 
both the present studenz and the future adult. 

• Obviously,' the concept of accoyntability as responsibility 
i& a very complex one, and the schools cannot be considered, as 
accountable in any simple sense 'to ajiy one group. (Thus making 
the issue or^*' problem much more difficult*. ) But whatever part 
of the' whole may be* in rtind,',the essence of concern is* not with 
'methods and means of accountabdlity but rather with tne ^natter of 
responsiveness.' Certainly, then, crucial in the objectives of 
accQuntabilit^ is an i^preasad responsiveness, of the educational 

system to those whom it' is supposed to serve. 

# • . * . ' 

Who then influences the system? To what pressures •do/. ttjj 
scIlDols resppnd? \ / . * \ ' ! ' » . 

• Students^ parents, and taxpayers hav^e the influence of cTon- • 
sumer and^ provider . Afe consumers, there li^ ,littl^e opportunity 
beyond individual and organized protest and persuasion to in- 
fluence pdfticy. As providers, the mdst^ effective action is the^ 
negative o le of withholding funds, (the* Ney Jersey track repord 
on ai^nual ]>udget and bond Issues is not a very 'impressive rone, but 
^simply a documentation of Jim-s point) with too little opport:Uitlity 
to distingiqish bet^en reasons bf tax burden and dissatisfaction 
with school perfoifmance. (You never really know when they.v.ot^ 
you down if it is because they .think thiey are paying too much .or' 
are .getting too little ^)^ • * 

The education prof essoin as a whole is, oi' cdurse, enormously 
influential, -^eachers* organization con trajct^ contain provisions 
about many th'ings Weyond salary andjrajcking conditions v (In one 
of my other livesy lihairjnan of the ^ard of trustees • of 'Jerisey 
City State College, we have a single contract with the. professors / 
af state contract. At the, state level .we deal, only with economic 



issues but at the local colle'ge^We deal with all the other 
issues. ' It turnecl. out that all the other issues are economic 
issues ^as well. Its hard to separate these two^ but the point 
that Jim is making here is that teacher organizations in their' 
contracts and in th^ negotiations haye a great deal of say- ,// 
well; beyond >that ,of influencing salary or working conditions'* of 
/the teacher.) The profession itself for the most part governs 
the .accrediting* agencies. Education lobbies exert great in- 
fluence on state ^hd federal legislation. (Everybody knows how"^ 
we^Ll vocational education lobbies work ^ for example.) Schools 
of education prepare the teachers and administrators who run the 
schools; they prepare also many of those in state and federal 
education agencies. Crganizations. .such as the AFT, NEA and AASA^ 
APGA h^ye nationwid.e constituencies and wield tremendous political 
power. ' ' 

Local boards of education have^the responsibility for the 
immediate oper-ation of -the school s^ and th^ir policy decisions^ 
though shaped within the limits^of state and federal laws and 
.policies and relationships to the profession/ in large measure-^ i 
determine the nature of. Iqcai educational opportunity. (The 
point .here is that the lop^l boai:d.^hai| a great deal to say about 
how you will serve the schools . and in what ways you might? be 
held acdoun table. ) 

Trie state has' the legal responsibility and authomty for 'thjS 
provision of education, (*^s outlined in. the Constitution) and 
its fiscal, regulatory and superxjisory functions prqfoundly— * 
'affect the quality of local perfomance'. 

The Federal government (even though it has only 5 oi 6% 
of the overall investment in education) because >of trie nature and 
.scope of present conditi.ons and trends in education is -assuming 
a mu9h more direct and influential role. (Ypu know, if 'we were 
talking 10 or 12 years ago we would barely ^'now the U.. S. Office 
of^ducation exists, but -now we firfd ourselves supporlbed by 
Title IJI or Title VII or what ever .it may be.). 

j Among all these interests bearing upon the schools, seeking 
to influence t'heir performaiiQe, whefe is the push that gets 
actipn? WKeii^^e is the force that can make accountability a work- 
ing reality? ' 

^ In my judgment, it is in government and the profession. 

Even with better means of exerting their loifluence and 
wielding their power ,^ the publib has dnlj/a .limited role... This* 
is n6t to- underestimate the Importance, ofl the public's role. 
Indeed, the public has to become more aggressive and unrelenting 
in its insistence upon pferfonrtance aS^ a measure of quality.. (The 
argument is not on perjCormance so much but on how we measure that 
performance.) 
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A promising note for the future of stronger and more active < 
public concern for education is the increasing interest of youth . 
in working for change within t-Ke system . and in gaining .^rea^er 
support. It was highly encouraging ±o me to have a student at 
Princeton consult me last Spring on. the feasibility 'of organi- 
zing a national youth movement in support of educ§itipii. What 
t)etter group .to reinforce the drive for accountability!. I 
strongly encourage such a movement and urgfe all concerned youth' 
to take part. (Jim Allen was a good friend of kids.) Not only , 
can the young bring .the insights of immediate experience but they* 
can also serve as a constant reminder that it is the product. and- 
not the process that mu'st be our constant concern. (The product 
in this case means the Kids.) ' ^ 

But even with the maximum exercise of their po;^er, the public 
can only deserve artd demand action. (they can't provide) — govern- 
ment and the profession have to -provide it, and the real push • 
myst come from these sources.* • - - 

It' cannot be ju$t gently pushing.^j>i^polite nudging For too 
long a kind of protecjitive pussy-footing, both in government and 
the profession, he^ ciharacteriz^d efforts- to jDr'ing" about real, 
change in education.} In seeking to avoid stepping on sensitive 
toes, we have too of^en succeededMn protecting the profession 
at the expense of t^^ children and youth — a true silent majority . 
if th^o:^ ever was onfe, just now beginning to rafse its voiqe 
and assert its claims. ^ 

In a broad sense, the edueational responsibilities of ^ " 
government at the state and federal levels and those 6f the pro- 
fession divide, with much intetacting and overlapping, into that^ , 
of the responsibility of gover^^im^nt for creating* those condition^ 
in which good education can flourish yrftid that of the profession 
for produfjing good education within* i^hose conditions. 

The ability of state and federal governments to cre'ate con- 
ditions favorable for good education will be greatly strengthened 
by three actions which would be immedia.tely gotten underway.^ :The 
first is a complete overhatil of the patterns of school finance 
involving at the federal Ijsvel a combin^atijbn Of consqlidate^w cate- 
gorical aids and' of broadly based general aid, allocated on an 
equalization basis, and, at the state leVel, assumption by the 
state of all, or substantially all,'. of the local costs of ele- 
mentary and secoiidary education. A recent Califprnia Supreme' 
Court decision has re-emphasized the imperative need for the 
states to *drasj:ically overhaul their School finance patterns in. 
the direction of full state assumption of. school costs.. (The 
way we presently finance education does* no^t provide- equal op- 
portunity tor all, and this fact- is going \o have a profound 
effect/ whether we are worried a"bout hbw to place kids in college 
next year or how we are going to get them readmitted to class. 
There is going to be a significant change^ and I thinj: JimjSignals 
it). . ' . . ^ ' 

The'second deals with structure. At the federal level, 
education should* be elevated to Department status with a Cabinet- 
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level Secretary and national advisory cCnunittee. { 

Department rank would ensure the sharper focus of greater 
visibility arid high status, andean advisory council would provide 
the opportufiity for advocacy of education's concerns, unhampered 
by undue influence of partisan politics or special interest^. 

' The council s^iould (as Jim Allen would see' it) be bi- 
partisan witri its members appointed by the President for stag- 
gered terms of sufficient length to prevent exdfessive dominance • 
by any on6 President or political party.. Its^oasic role shoul^ 
be advisory, not administrative. A's an established body, not 
•tied to the four-year election cycle, it could provide a -con- 
'tinuity that could help- to mitigate the disruptions of changes , 
in Administration. It could enlighten the nation by making an 
annual report to th^ President, and the Congress on the state of 
education. (A goodipciint here is t^at thfe U.;- S.- Oftice^f 
Education w^s estciblished by the Congress to report to tpe Con- 
gr«S« annually on the s^tate of the American Education^^^y^^he 
Office of rEducatioij^is over 100 years old, and it hp^^nevex re- 
ported on the^ statd or condition or Ameri<:an education td any- 
Bbfy.) It could act as a conscience for the, people cjf the 
country by ♦^both -ref lecting and 'arousing^ concerii-'foi/, the- well- 
being of educc^tion. ' ^ ^ y ^ . ' ] 

At the st^fe^ level, a strengthening of structure is even 
ifc^re essential^ especially ,in £^he light of the trend toward 
'proposals for ^ull state funding. As the 'statQs are not..orr- 
ganized and managed they are, in varying degrees, unprepared ^o 
encompass the expanddid di^mensions of their educational* tac^k or 
to' deal with the broadened concepts that now de'fihe their role. - 
J'hus, each state must examine its o\^n capabilities, and undertake, 
to whatever degree may" be necessary, ah overhaul -of its .arrange- 
ments for the governanoe of education^ Expanded federal support 
for education should incorporate incentives for .modernization of 
state edaipation .structures. .^Perhaps the state education de- 
partments will not. look like ref lecti9ns of* a local school board' 
or' a local administrative unit of education) . . • * 

' Thp third action ►i^equired to aid government in its re- 
sponsibility .for creating^ the conditions for good education is a 
systematic program of research. and development.^ This^ aid" would 
be substantially furthered by Congressional enactment of the 
legislation establishing a National -Institute of Education, pro- 
posed last year by the President. (Which now has found its *way 
through some parts of the Congress, and as of about a week ago, 
the outlook is positive. "We may^ f or the first time' have something 
in Education which might be* considered akin to the National ' 
Institutes of Health. Federal xn^estment and fundamental research 
in matters of learning, in matte^rs of instruction, done in a 
positive way *and supported with sufficient resources, would aid 
air of us in seeking to do our jobs better.) ^ 

Obviously an adequate discussion of these actions would be 
^a speech in itself (and Jim 'has speeches I'm sure), but I cite 
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them because they are essentiaL to full effectiveness of the ^ 
governmental role"' in accountability,. 

Despite the widesprefad interest in accountability, 
( Jim Alien), doubt that it will advance on any large scale to the 
action stage without the determined leadership and construc- 
tive ,v focalized help of state and federal governments in both 
ways and means. (In some ways; he has let you off t^e hook, 
He said, "Now we will leave 4t up to the Federal and State. ^ 
legislation," but watch out, it will come back at you later.) 

j Government must assist in developing and making available 
the techniques and ijistruments of accountability. Government 
financing of education must recognize the need for account- , 
ability and use the pow^r of the purse as leverage to encourage — 
indeed, where necessary, to require — accountability • (AncJ 
jthere are schemes afoot that would reward school systems When 
productivity has been demonstrated.) 

^ Within the' profession there is growing support for account- 
ability, but* as is too often the case, it comes primarily from 
those who are already productive with support lacking where 
accountability is most needed. (You know, there are a lot of 
people who say they want to be held^ accountable as classroom 
teachers or tYte school principal' '^r school superintendent, or 
the* dean of education, bilt they ar e generally the people in our 
ranks who are already producing.) 

" ' *» . 

The resistance to -accountability is understandable'. In the 
minds of many, it sets up an image of excessive testing and 
measurement, inimical to initiative and imagination. It implies 
criticisr., generates def ensiveness , and leads to a kind of pass- 
the-buck attitude. (Once I made a speech to secondary school 
principals and said accountability is a school issue. I took 
teachers off the hook by saying it's difficult to hold a seventh- 
grade geography teacher accountable for»all those kids who pass 
through that door each day. I said the accountability unit in 
education is the school .and not t^e teacher. So I made it 
possible to pass the .buck from teachers to somebody else.) 

An acceptance of accountability requires a perspective that 
recognizes the indivisibility of education and concentrates on 
its purpose.' ^ 

This kind of perspec\jye has become increasingly difficult 
to achieve as the numbers to be dealt with, and the expanding 
dimensions and character of education, have produced an undue 
concentration upon the parts rather than the whole. 



Despite/the vast machinery of the system, hcever, educa- * 
tion's purpose is * still simple — the development of tHe intellect 
and the discovery and encouragement of abilities and talents. In 
attempting to achieve this purpose on such a large scale, each 
of the various* par.ts of the enterprise has itself become so 
immense as to demand a degree of attenrtion that excludes, or makes 



very difficult, a constant mindfulness of its place within the 
whol§. (Since most; of you are counselors, you might well ask 
yourselves, "where does guidance fit in this whole account- 
ability scheme?"- Is it just to. hold a counselor accountable 
for what happens to' the child who leaves school? Perhaps not,, 
but the que.stipn remains, what piece of the accountability ^ 
fabric &re' guidance and counseling servicas responsible for?) 

! In relation to this point, I read with much interest the. 
New York Times report of a speech by Buckminster ^'^11 ^ de- 
livered" at. a meeting of the American Association i. v»ms, in 
which^'he said "Wfe have so ^any specialized abilic^ . we can blow 
burselves to pieces,* but we have no ability to coordinate our- 
selves, I ^ee^our^' socis'ty as very .powerfully conditioned by its 
reflexes, with very, very tight ways of functioning. And "that, 
is d^ngerous-r-so dangerous that if man' does not stop thinking 
locally^ and make t^ie grade as a world tnan we may. not be' able to 
continue on tfiis planetvi". Th^ news report continued — "Citing 
the iieed for 'synergism^' thinkiijg in terms^ of the whole rather 
tiian.its parjbs, Mr. Fuller 'urged the development of a larger 
c^Jncept of-man who coi:^d think in such terms," 

Moj^e synergistic feh/nking within the 'prof ession can be a 
strong aid to a widespread acceptance of the necessity of account 
ability, (It^s not how we hold the teacher acco\intable, the 
counsi^lpr accountable^ the principal, or even how we hold the;^ 
pupil accountable; it^is rather that we a\i /see that wfe'are in 
concert/^one with each other, accountable for something,) 

• * . '' " 

By this time,' if there was ever any doubt in your minds, it 
must be obvious that I (th^t is, Jim Allen) believe the account- 
ability b^indwagon is on thfe move because accountability is the *. 
most promising Qure for many of education's serious ills'* ' 

The public to whom accountability is dpe is becoming more 
and more aware of this -possible remedy, (performance contracts, * 
.voucher systems, national assessments) and is »also much more 
sophisticated., and able to detect any attempts to substitute 
more of the same old. brew in new bottles. (What Jim Allen is 
saying is we' cdn^'t get by by saying, "We have lovely schools, 
come and see us ^ once a year,") 

The responsibility for prescribing this remedy and for get- 
ting it into action rests primarily yfith government and the 
profession — and if we faiix|iow to folXow through, to .make 
accountability a reality, thfe^^esultihg loss of confidence (on 
ppirt of the public) and withholding of suppor,t will make the 
present doubts about thp ef f ectivenes^- of our 'Schools look like 
a veritable avalanche of approbation. | 
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ACCOUNTABILITY: IMPLICATIONS FOR GUIDANCE AND CAREER 

DEVELOPMENT 



, ' ' John 6, Odgers ' / : \ 

'* ^ ' * ^ ^ 

; Probably there is no betteP way to start a paper on account- 
ability than to admit that ^actual facts' do not always correlate 
positively with appearances*, I'm reminded of the do-it-yourself 
addict who heaved a sigh of relief and reached in hj.s shirt 
pocket for a cigarette as he sat in the corner on the new car- 
peting, having just pounded the last tack. Not finding the pack, 
he glanced across the room and, sure enough, there in the 
middle of the carpet was a small Iximp. Disgusted, he went to 
, the closet,' got out a ball bat and flajbtened the bump so that 
no one would know. Completing this* task, he walked into the 
dining room only to see his open cigarette pack lying on the 
talkie. Almost at the Same instant from the family room came 
his wife's voice:- "George^ have ypu seen* the parakeet?" No—* 
things are not always 'what they appear to be. Sometimes they •re 
W9rse — sometimes better. -But it's important to know how close . 
you are coming to meeting your objectives and at what-^cost. 
Cost in terms -of money, cost in terms of time and energy. ^ - 

As a sta:|^ter, I would like to point out that the concept of 
accountability isn*^t -as onerous to people in education as mdny 
outsiders -think.' Although maiiy educators are thinking in terms 
of evaluation rather than 'the broader concept of accountability 
(which studies ef fec;t:iveness in combination with efficiency) , 
more and' more educators are designing program change to buil^ in 
accountability — ^^both incrementally and . summatively . - And this 
involves advance planning. I want to come back to that later. 

As never before in education, there seems to be a climsite 
for change; — a national discontent both. within and outside .the 
schools;' dissatisfaction with the state of traditional educa- 
tional programs and the effectiveness of their impact on children 
Committed teachers, administrators, and boards of education 
recognize that they can't survive without maintaining ongoing 
accountability. Many districts are seeking adequate leadership 
so they can become more effectively accountable. Not only are 
teachers, administrators, and boards of education showing more 
concern; parents, and other taxpayers want to know what will be 
achieved with their dollars in terms of relevant program outcomes 
Dr. Allen emphasized this point throughout his speech. 

^In preparing for today's conference, I was asked to consider 
the concept of accountability from the poii^t of view of the 
administrator. Having been a guidande administrator for the last 
quarter century, and this being ^a guidance conference, I am 
naturally assuming that I can react with the school guidance 
prograJn uppermost in my mind. I would like first, however, to 
ma^e a brief and undocum^ented review of the development' of the 
concept of accountability— and then, expand on a point or two 
empTiasized by Dr. Allen. 
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Historically, systems theory had its origins in philosopliy, 
and remained, quietly foi? generations as a tooT of philosophers 
and logicians. Gradually systems applications were taken ovei* 
^ bif. science. More recently, McNamara put systems into national 
defense. During the last fifteen tp twenty years business and • 
in<\u^try have put systems" theory to work effectively in produetiqn, 
finance and management.' Just now are ^e ready %o put systems 
into education — recognizing that this must be done without de- 
humanizing the process.. To accomplish^ this requires new com- 
- petemries. To accomplish it requires thorough kripwledge both. of 

the culture and of huirian* developjpent—- and the ability to maintain 
. balance between these. A .good systems approach to education 
(educational administration) mu;5t £?teter the development^'of 
f knowledge, feeling ahd skill in balance — to motivate and to pro- 
vide the background .'for mature adult self -social fulfillment. 
^ Uncommitted knoy^ledge. is most wasteful . *Commdtted incompetence . 
is dangerous. Education ne^^ds a rational. dfeSign and people who. 

believe in it and will work to make it work. 

• ». • 

In the paper he prepared to present to you toda}j. Dr. Allen 
used the following quotation from a New York Times report on a 
June 1971 speech by Bxickminster Fuller: "W^j have so many 
specialized abilities we can blow ourselves to pieces, but we 
have no ability to coordinate ourselves." In a January 1971 
presentation to the Convention of the National Association of - 
Secondary School Principals, U. S. Commissioner of Education, 
Sidney P. Marland expressed this same concSern, .as it relates 
education, in this way: . . education's most serious failing 

is its' self-induced, voluntary fragmentation, the strong tendency 
of education's several parts to separate from on^ another, to * 
divide the entire enterprise against itself." He went on to 
state that '•the most grievous example of these intramural class 
distinctions is . . . the false dichotomy between* things academic 
and things vocational" and to suggest that,.' "as a firist step . ; . 
we dispose of the term vocational education, and adopt the/ term 
career education.*" 

These last comments by Dr. Marland have dif.ect significance > 
for your school guidance programs. Before examining/ their im- 
plications in relationship to accountability^ in guidancje program 
administration, howeVer, r would like to bring yjpu the opinion 



of one additional authcTrity on accountability. 



In an article in the December 1970 Phi Delta ^appan (a* 
special issue on Accountability), t)r. Henry S. Dyer, Vice 
President of Educational Testing Services, described account- 
ability, as it applies, to ah individual school, ip the follow- 
ing manner: 

-'"1. The professional staff of a school is to be 
- held collectively responsible for, knowing aLs much as it 
can (a) about the intellectual and personal-social 
development of the pupils in its charge and (b) about 
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the conditions and educational services that may 

facilitating or /impeding the pupils ' 'development,/^ 

1 " '"2, The professional staff of a^'^school is to be 
held collectively responsible rfop using this know- 
ledge as best it can to maximize thte development of v. 
its pupils toward certain clearly def,iicpd and agreed- 
upon pupdt performance objectives, * . 

"3. The board of education has a corresponding 
responsibility to prpvide the means and technical 
assistance whereby the staff each §choQl*can acquire, 
interpret, afrid use the information necessary for 
carryj^ng put the two fore^ing functions. 

♦ 

"I emphasize the notion of joint accountability 
of the entire schoQl staff in the aggregrate — the 
^, principal, teachers^ specialists — because it seems 

obvious that what happensjto any child in a school is » 
determined by the multifefide o£ transactions he has 
' ^ with many different people on the staff who perform * 
differing roles and presumably have differing, impacts 
^ on his learning, which cannot readily, if ^ ever, be 
disentangled . " * 

^ Dr. Allen emphasized^ iiv^is .paper the importance of the 
product, the youngster, the people we turn out of our schools. 
But, I think you will notice immediately after emphasizing pro-- 
duct he identified need process changeiS to make a better pro- 
duct. I think we need to keep this iri mind if we are focused in 
^on producty but we have to pay definite and earnest attention to 
, process to improve product. * . v 

I would like -to use my last few minutes^ to relate my prior 
remarks to guidance program* administration by quoting to you a 
brief statement which I made recently to the representatives 
from six school systems in six states which hay^ been carefully 
selected to help commissioner Marland's dream of careet education 
come true. As many of you know, the U. S. Office of Educat?.on 
has entered intd contract with^The Center for Vocational and 
Technical Education at the Ohio State University "to develop, 
test and install a career . education "sy.stem with the** potential to 
revitalize the total educational program; of a public school 
system by structuring student experiences around career lievelop- 
nient Objectives.* ^The Hackensack, New Jersey system is^one of 
the local educational agencies participating with us in this 
important project. y 

If^ we are to develop an teducational system which will 
achieve this obtjective, changes wilJL have to be made in the 

^CVTE, A Comprehensive Carger Education Model, 1971, p. 5 . - 
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typical school's guidance program to majce it an integrated 
and basic component of the total eduction strifq^ure with a 
staff and a board of. education which meet the criteria -outlined 
by Dr. Dyer. Here is my message and, charge to the six CCEft 
schools: , . . * 

"Ask any sohpol counselor the major. objective. \ 
of his guidance program and he'll probably give you ' i 
an answer something like this: 'To help every boy \^ 
and girl develop into a mature, responsible^ self- 
directing adult.' Ask him to be more specific 
and he'll probably come up with a series of develop- 
\ TOental objectives which, with a little editing, * 

sound something like this: ^ *^ . 

"To help each boy and girl 
\ . to grow, in self-insight aud self-undetstanding ~ 

.to grow in knowledge and appreciation of his- 
\ en/ironment: ^its opportunities., its limitations, 

\ its demands 

io develop, decision-making skills 
/to establish personal value^^ and standards against 
'which to weigh alternatives when decisions must be- 
ma4e • , • 

. to acquire the interest and motiv^ition prerequisite 
' to action 

4 to make plans and take action on them 
. to assess his successes and failu res a nd to |nodify 
his plans accordingly.' ^ , 

So much for the counselor's stated objectives! 

ft 

Now ask him to summatise a case and he will most likely 
describe a boy or girl in need of help to meet a crisis 
situation or solve a pressing problem. Analyz6 |the 
walk-in traffic coming to his office, or the pupils re- 
ferred by teachers and the odds are you'll find kids 
with problems. They aren't there for development, 
they're there 'for help. ^ 

Something must be wrong with our system if -'our guidance 
program gives lip serVice to objectives stressing a 
developmental approach but counselors find themselves 
spending most or all of their time tackling crises. 
The frustrating truth is that if we don't plan develop-- 
mental guidance programs, to help students develop in 
such a way that crises are less frequent and less seyere, 
we may be working for a losing cause. (Our program must 
be systems oriented and accountable.) 

In order to set the stage for a more balanced and 
effective guidance program, let's admit that most 
counseling is crisis-oriei^ted. But let's also face 
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the fact tK^v'^J^ost kids don't nfeed frequent counseling. 
What: t|^y do yieed is regalar help in the nontial , process of <^ 
career development; which is not typically provided on X ' ^ 
one-to-one basis . In fact, ^ career guidance is^ usually 
more, efficient and effective' "if provided in a group setting, 
oft^.'ia. the f^rm of iristruction. 

The^ coifunents %fi no way deny the impi^rtance of the 
counseTing process, .either as it:, relates tcr crise© in m >^ 

^the lives of students, or, ^o the need of ma^ny students ^ 
for confirmation of personal plans\ or for*cj^lues ti 
alternatives. They do, however, ^ve stroiig^ suppont 
to the fact that, in an accountability-orien^ted sclrool 
system, a significant portion of /the professionals 
and cupjk>rt stfif^ manhours assigned ^o the guidance ^ * 

^program may well be assigned^ toe duties other thap * 
counseling students . , 

A major responsibility of the guidance program in\such t 
a system is to ser\^e as a facilitator and change agent 
in relation to the career education program', the total 
curriculum, and the community. Appropriate guidance 
staff members will serve as: ' ' 

. Consultants to teachers (helping teachers 

understand. students better; providing 

teachers with occupational and labor: 

market informat'icn, etc. / 
.. coordinettors of group guidance programs 
. counselors to parents 
. coordinators of community resources 
. managers of the school placement program 
. influencers of the curriculum 
. referral agents to special services 
. coordinator^ of (or consultants to) the 

school's testing and pupil appraisal program 

In the Comprehensive Career Education Model, more and 
more of what the counselor \has traditionally considered 
. his responsibility is bein^» incorporated into the 
curriculum, including developmental approaches to in- 
creased self-understanding, \knowle&ge of the w6rld of 
work, and the development^ of decision-making skills, 
fin this context, the courfselor's responsibilities 
often change j^om a major' emphasis on direct service to 
students to the role of resource agent to other ^staf f 
members .» (A' significant part of this new responsibility 
may involve- assessment of hoW well program components, and 
the program as a whole, are meeting'^esired outcomes.) 
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Definition and assignment of specific functions within 
the guidance program are a responsibility^ Qf each CCEM 
school and will normally be influenced by the educational 



level served, 1^ th^ nature and needs of the student 
body, and any .specialized objectives* o£ the school 
in question (e.g./^a vocational high scjiool) . • 
(There will,.! am certain, be an increasing' emphasis * 
on the use of para-prof esslonals, . 

The development of a full-funotioning "gui;dance 
progra&n will be s4inp#ified and enhanced if tfie 
program is built around a conceptual framework 
which recognizes that the components of a** total \ * , 
prograon fall 16^ically into two groups: the ' 
Content ^( or supportive)" components, V/hich arp systems 
oriented a'nd vy^ch normally do not involve direct^ # 
counselor-student contact, and the Delivery System ' ^ 
dimensions which -'involve direct" servicet^rby guidance 
operatives (supervisor,, counselor, or para-professional) 
to students, parents, or staff members," ' ^ ' 

r 

Tjiese' basic components may be fur<ther defined, by 
type, as follows: ^ 

Content or Supportive Components: ^ 
Pupil Data System 

Careeif Information System * 
Educational 

, Occupational- Job ^ 
Labdr Market \ ^ 

Resource Coordination System' 
Guidance Program Planning and 
Evaluation 

Delivery System CompbneAts: ,p 
Student Services ,v 
Group Guidance 

Counseling * 

Referral 
^ Placement 

Staff Involvement ^ 
Parent Involvement 

So looking at the program this way we have a base for • 
checking up on ourselves or a base for accountability. • 

Special emphasis should probably be plaqed on the 
Placement Services Component. / 

I believe that we all see as the single dominating 

objective of the Comprehensive Career Education 

Model the development by each Local Educational 

Agency of ao educational program of a nature and quality 

that will assure every student placement in an 

appropriate next step beyond high school, either 

in ^employment or in a Continuing educational pursuit. 

The existence of a good educational program, however. 



does nofe^'^guarantee the placement of its end products • 
The CCE5I scjiboi must be geared up, with a .formally. 
^* ' « organized, placement program, to he lp\ every 'student 

tak« th^t next step when he lieaves .sch6ct/ ^'whether * , 
he.be exiting *to a job or to farther "edpfcational ' - ' 
pursuits arid, whether^-he be exiting'^ciim laude or \ ^ 
Sana diploma.^ * / • „ " . . ^ 

Possibly the best parting shot I can- take wjien' exfuoining 
,the concept of a^c^puntab'ility as it relates to guidance progreinv 
^ a'dministration . is to remind- you that the concept placed emphasis 

QP planning for fesu^s. ^ x thifnk t4vis is it, . We' must plan 
\for results if we are going com^ up, witii results in the end, 
Yo provide adequately fpr ongoing accountability any program. 4 
must: * ^ ' * * ' - 



be mission oriented ^ • ' " 



. bevdesigned to flow from stjurt to' finish 

. be designed to facilitate program management 

and ^monitoring ' . / 

^produce alternative methods to th^ exterjt 

ne.eded \ 
..provide for evaluation and feedback. 



V 
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The CCEM Guidance and Placement Program, Fleinarks made ,by John 
G. Odgers, Specialist, Guidance and Placement, at orientation 
meeting of CCEM participants at The Center; for Vocational^ and 
Technical plication. The Ohi^ State University, September ,22, IS 
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My thoughts t€^day are very much of Jim Allen, He devoted a 
considerable part of his time this past year as a senior advisor 
to ETS. Like most jJeople in other spectors of American education, 
we at ETS will miss his courage/ his compassion, his dedication 
to democratic ideals and values. ^ , > - 

Last tiine I saw him we were partners in a tennis match* 
His vigor and confidence on the court remain vivid in my memory 
as a reflection of the energy and ?^al to which he ^applied 
himself to every. activity . It is a ^ood memory to hold on to. ' 
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EVALUATINfe GUIDANCE— WHY r WHAT, AND HOW 
' Martin Katz 

S . J - \ ) ^ 

• My /talk today will deal with three major questions about 
evaluat^^ji lof school guidance progreims; Why should we evaluate 
them? What, exactly, are we evaluatiq^? And, finally, how can 
we go about doing evaluations? * . * ^ 

The first question. Why evaluate, is the easiest one to 
answer. One reason is to do our work better. Another reason 
is to convince others that our work is worth supporting. A third 
reason I will hold in suspension until after we have talked 
some more about why, what>'*and how. ' ' * / 

The first reason — self-improvement — reco)gniz.ds that a . ' * 
counselor is accountable to liimself . EvaluSition by the counselor 
ii> terms of his own standards, expectat:j^ns , and concepts is a 
continuous feedback loop of the sort we^ll*use»to monitor most 
of pur efforts and try to itaprbve them. The emphasis in this 
evaluation for imj)rov^ment is on processes and short-term prp- 
ducts. At a very primitive levfel, th"^ counselor watches what he 
is doing .while ^he is «)ing it, makes some observations 4bout 
immediate effects, and takes corrective action as indicated ^ ^ 
•There's no sense in waitiffg for long-term outcomes if you* know 
you have to make a change now. For example, a speaker oi) a pjiat- 
form — like this — can a^nse' whether anyone is listening to him * 
or not. hot, he hds to' do something <nffer:ent — tight away — 
talk louder or softer^ speed up his delivery or slpw it/dbwn,* say 
something piew or^ maybe something mqre famj,liar. ^* 

■ /^'^ ^ ' ' ' ' 

This is evaluation of micro-actions — before they have 
aggregated and agglutinated into a macro-program../ it deals with 
the "necessary but not sufficient" corCaitions for su9cubss. If 
there is anything that can possibly be accomplished by* speaking, 
it. can't be accomplished unless someone is , listening. V 

If a counselojf* announces office hour? and sits back to wait 
for students to coine in^rand no onoL comes in — thari^he knows it's 
time to try sometiiing e^Lse^ If he puts occup^t?ion4i information > 
into a file, and no one uses it — then knows it/s not doing any 
good,^and a new appj'oaich is required. . ' , 

This mention of "soma(thing elsi" or a "new ap^Jroach" 
suggeats that evaluation involves a choice betwfeen alternatives. 
it helps to' have a big pool of alternatives available. If there 
truly were only one way to accompli sfr an objective, evaluatipn 
of process would^-be futile. Occa»ionklly , we ^eep on doiKg some- 
thing — even though it Ideesn't vy^rk — b^ause. we^^can't think of ^n . 
alternative. Creativi'^y in counselors may often take the form of 
seeking -^nd finding other ways to do/thingi?^ when our current way 
does«'t^work. 'Creativity may be foptef fed^ and istimulated as 
counselors make these ongoing informal evaluations more syste- , 
matic,* more structured,, more explicit. My^cJol league, Henry ^Dyer, 
calls this kind of , simple systematic* inv^tigation "shirt&l^eves 
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researeb." It may consist of no mpre sophisticated data-col- 
leCtion than counting. For/ example, how many students used the 
occupational- information file? Just formulating the question 
may be enough to indicate what kind of data are needed and how 
they should be interpreted, at , least for this most primitive 
level of process evaluacion'^. 

Evaluation of process leads" naturally to evaluation of 
product. If' the students are reading the occupational information 
, materials, what good is it "doing th^m? Are they learning some-- 
thing important, and. useful' for their career decisionrmakirvg? 
if they are coming-in to- talk to the courv^elor', wha± difference 
is it making? What contribution is eagh procedure^ or facility' 
giaking to sonSe outcome, or pf'oduct? To gain some objective, 
how^much time should a given studjent spencf on each procedure? 
How much of the talk or of the reading could you delete without 
affecting' outcomes? John iWajiamaker, the department store mer- 
chant,, once^s^iJ, "I ^ know that half the money I spend on adver- 
tising^ is Wctsted. The troujDle is -I don't know which half%" 
Maybe half •'•'the time we spend talking to kids' is wasted • * (Runkel 
did' a s^udy in Illinois high school^ that showed no relationship 
between such process variables as frequency of. stucient-counselor 
talks and such cri.teria*as students*' information abov^t chosen- 
occupati9n and the appropriateness of curriculum choibes to 
occupational choices.-) '>Can we devise studies that open ufr-the 
•'black box" gf the counseling interview and't^ase out the elements 
thajb -are effective"? — effective., that ^is, for which students » 
under which- circumstarices accomplishing which objectives? We. 
^rbbajDly can't do this in ^shirt sleeves" — we need to put "bn the 
/research specialist's coat for ^va^luations of that complexity. 

• But the first step in such evaluations-^-the step in which 
purposes are stated — is ohe that the counselor cah take"tind 
should certainly- want to take. Others may not agree with his 
goals — with what he)say£, should be the product of this work. But 
he hais to spe^^X 'out What* he is'trying to do if he is going to be 
accountable to himself' for butccSmes. I am not saying that he will 
readily find. the oppo^tOnity to check out how well he accomplishes 
;his long-range goals, his ult,i<mate product. But at least he 
has to have the** long-range objectives conceetlialized in or^er to • 
define intermediate ajid shb'rt-range objectives that are logically 
aligned with them. In^ brief,* the*, improvement of processes im- 
plies that purposes and goals are known. Others may not neces- 
sarily agree with them — but the counselor shall say what his* 
goals are. 

^This first, step in*4^he counselor's accountability to him^ 
self* is also the' £i/rst step in his accountability to others. The 
jaoTfi explicit hfe can tnake his own objectives for guidance and his 
stajidar^s for judging accomplishment, the more clearly he can 
perceive the demancis, ^expectations , and standards of others to 
whom he is accountable — stu^ients, theJLr parents, administrators, 
cJthei;^ scl|Gol staffs the 'community. Enlarging in this way his 
consciousness of agreements and diffeijences between his own con- 
..cept of his rple and the concepts others have of it, he is better 
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prepared to Tiegotiate with these others—to build on areas of 
agreement and to try- to reconcile differences, or at least in- 
crease understanding and tolerance of differences ori all sides. 
By knowing and communicating his objectives, he can influence 
the nature of the accounting system which others may use in 
evaluating his work. He can help dij^ct the traffic not . just 
stand there and maybe get^un-^over . 

In the speech he would have delivered this morning, 
Dr. Allen used a different metaphor to express his hiyg^h -hopes for 
public accountability. He called fMblic accountability "the 
most promising cure for many of educatior^'s most serious ills." 
He warned, however, that the public is becoming "sophisticated 
and able to detect any attempts to substitute more of the same v 
old brew in new bottles." This expression brings to mind an 
episode described by my wife on her return from the weekly shop- / 
ping she does every day at the superme^rket . In the pacing lot, 
she saw a wor-*n she knew to be pregiwi^t suddenly slump over .the 
steerii^ wheel. Fearing an "emergdndy," she ran t© offer help 
ar.d found the woman doubled up not in labor but in laughter. It 
seems she was en route to visic her obstetrician, who had told 
her to bring in a urine specimen. The onli^ container* available 
at home had bepn an empty whiskey bottle.. While she was in the 
store, someone had stolen her whiskey bottle. Oui; moral is that 
the new bottle labeled ^gcountability will not fool many people 
for very long, if tfie^contents are the same old *bleep--which has 
frequently been used in evaluations — counselor^^'student ratios, 
or hours of graduate study completed by counselors, or size of ^ 
the occupational library. ^ * . 

^ ' *\i 

Going from scatology to e'schatology , we must Expect — as-^ 
Dr. Allen has warned — that public evaluative judgments will be 
made of guidance programs. Since the beginnings of NDEA,' gui- 
dance has enjoyed a favored status.* Under JSPEA suppo^it, guidance 
programs were established at many schools that had previously 
had none. But after the mid-60' s, NDEA support fell off,- and the 
burden- fell lieavier on local school districts. In his recent 
book, Eli Ginzberg recommjends cutting off all tnandated federal 
aid to guidance., :ie urges that- the issue of support for guidance 
programs be decided "not in the halls of Congress but closer to. 
home." In other words, he would put guidance needs in the pit 
with other educational needs. The magic claimed for accountabili 
ty is that in Lessinger's words, "resources and efforts are re- 
lated to results Tn ways that ?re useful for, policy-making, 
resoutce allocation, or compensation." TSiua the decision-makers 
at federal and local levels want to examine togt->ef fectiveness 
so that they can make decisions about deployment of resources. 
The present Commissioner -of Education, Dr. Ma^rland, has recently 
made /a commitment to support model career development programs 
with a strong guidance component — implemented by $9 million 
allocation for 1972. His directive requires emphasis ori '^careful 
measurement of, student outcomes in relation to the' treatments. " 
It also requires cost infomatipn on each component. Finally, it 
calls for "third party evaluation." So\we see that even the 
supporters of guidance do not exempt guidance programs from 
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judgments. These judgments, however, are Ibwer case and plural. 
They should not be mistaken .for the Judgment Day, when presumably 
the purpose of the evaluation will be perfectly clear, the 
c^'iteria sharply defined, and the measures absolutely reliable - 
and valid. 'The present-day judgments, in contrast, will be 
fallible: we see no clear consensus on purposes, there are 
sharp disagreements on fuzzy^criteria, and measures that* have been 
developed so far appear to have validities that are, at best, 
indeterminate or modest. 

This brings us *^to the question of what we are trying to 
evaluate. There seems little prospect in the immediate future 
of convincing the public — or even fourself that any of the fol- 
lowing direct questions can be answered definitively: Does 
■ guidance work? Does it achieve its' goals? How well is the 
guidance program at your scfiool doing? Are children getting good 
^* guidance? What difference is it ^making in their careers?^ Are 
the programs worth what they cost? Is the money they cost being 
used efficiently? Should^^ the guidance programs continue to do 
what they are doing? - ' , 

^Tumin has called ey&luative questions like these the "fool's 
questions" — "because they are absolutely right to ask and im- 
possible to answer as put." These are the big questions that% 
research and evaluation studies have nejver fcTeen able to answer. 
At' l^ast not unless one fragments each of these questions into 

'>sub-questions, defines each fragment in opeJratiqnal terms, sajnples 
from the new sets of ^questions that are thus generated, and 

♦ identifies relevant observations or* measures with the expectation 
that enough such observations can be combined tp represent., a 
• ^"^ facet of each little question, and that enough answers to little 
questions eventually allow us to assemble some kind of inference 
about' one of the fragments of a big question' — and so on. 
* ^ !>• " • . ^ ' ^ » . ^ ^ ^ 

Let's take an illustration. We ask a big question; are high 
school students getting goo4 guidance? Let's define a ^ub- 
question: Ar^ they making their career decisions wisely? This 
,sub-que5tion must* be sliced up into smaller and smaller questions 

'before we qan begin to answer 'it. .Recent studies have atten^ted 
to elaborate a construpt called 'Vocational maturity," and ask 

'whether students have gainegl in vocational maturity i One in- 

. dicator of vocational maturity might be, are they seeking^ occupa- 
tional ^infftrmation? One of many ways in which they mignt seek 

'occupational information is through reading printed materials in 
th6' occupational information library or fileis. Aha, now we have 
something we can observe or measure. We can count the uses made 
of these materials, we can- ask students what use they make^ of them, 
we ^ can test stCidents on the information contained in them^ Does 
this kind of observation or measure tell us whether they are 
making career decisidns wisely, ^and wht^ther they are getting good 
guidance? How many little questions like tnismust we answer in 
order to make an inference^ a^ut the big question, "Are they making 
career decisions^wisely**? ot "Are they getting good guidance?" 

Am I lacking in the reverence that is usually givenby ^ 
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evaluators to "behavioral objectives"? Do I imply that defining 
and measuring behavioral objectives is not adequate for evalua- 
tion? Just so. Focusing exclusively on behavioral objectives 
can lure us into rationalizing the inclusion of behaviors just 
because they are easy to measure. Often the use of such be- 
haviors and their measures in evaluation tends to impoverish 
rather than to enrich practice. Teaching to the test makes us 
lo^e sight of the big question/ the. fool^' s question. Guidance 
id not the only field in which this problem occurs. Even the 
"hard curriculum" areas face it. For example, an ETS colleague, 
Sheldon Myers, had criticized current sta'tements of behavioral 
objectives for mathematics in elementary grades on the basis of 
their "great specificity. The unfortunate consequence of this 
atomization is that the i'nterrelatedness of mathematical concepts 
is lost and the statement is a tedious list of very trivial low- 
level skills." 

L6e Cronbach has pointed out that specific behaviors can 
and should be used' as indicators of constructs but not as the 
definers of those constructs. It is the constructs, the network 
of relations or characteristics, that are 'cnaci.al to evaluation — 
not a. single specific incident of behavior. "The operationists 
who want to equate each construct with one indicator," he says, 
" • . . are advocating that we Restrict descriptipns to state- 
ments of tasks performed or behavior exhibited and are rejecting 
cons.truct interpretations. . . . The writers on curriculum and 
evaluacion who insist that objectives be defined in terms of be*-, 
havior . . are denying the appropriateness and usefulness of 
constructs. " 

Let's point this problem up by assuming that you are working 
under a performance contract. You are to be paid according to 
the "results" you get. Now how. are results to be measured? You 
name one objective of guidance, as helping students make career 
decisions wisely. . You invoke the construct of vocationaTTtfaturity 
You assume *that information plays a role in this. You may reason, 
as I v^rote some years ago: 

^ Decision-making . . . may be .regarded as a 

strateq^^ for acquiring and processing informa- 
tion./^^ If a decision is truly to be made; if 
it .ic not a foregone conclusion, it must in- 
volve^ some novel elements. The person con- 
fronted' with the problem of decision-making 
either does not know what information he • 
needs, does not have what information he 
wants, or cannot use what information he 
has. Thus, the pressure for making a - 
decision • creates a discrepancy between the 
individual's present state of knowledge 
(or wisdom) and the state that is being 
{ demanded df him. 

The role of* guidance should he to reduce the discrepancy 
between a student's untutored readiness for rational behavior and 



some hypothetical ideal state of knowledge and wisdom. So the 
appropriate criteria for a given program designed to retail in- 
formation might be: (1) Do students know what information they 
need? (2) Can they get the information they want? (3) Can 
they use the information they have?" 

But when all this language gets translated into specific 
measurable behaviors for a performance contract, the contract 
may caal for a questionnaire to be given students on the extent 
to which they use occupational information materials, oir^a count 
of such uses, or a test of knowledge of facts about occupations. 
Would you as the contractor then attempt to develop in students . 
a general competency in the strategy of information-processing? 
Or would you — as the Texarkana contractors are alleged to have^ 
done — find a more direct route to raising scores on the criterion 
measure? After all, students can be induced in many ways to take- 
materials out of a library, or to respond in a certain way on a 
questionnaire, or even to~ memorize some facts. They would not 
need a "guidance program^ for this — just, if we wanted to be 
crass about it, a little coaching. One can raise scores on such 
criterion measures withdUt affecting the outcome that is of real 
concern. Such an increase in scores would be no more valuable 
than, in Thorndike's phrase, boiling the thermometer ta heat the 
house . 

The ripple ef feet '^f studies that use such measures qf 
specific, behaviors is another problem. By the time the study 
report gets cited in the literature, the specific behaviors and 
measures that underlie the findings are often forgotten. A verbal 
suuTunary of the conclusions is quoted and requoted: "This treat- 
ment significantly increased information-seeking behavior of 
students and thereby contributed to an improvement of wisdom in 
decision-making and again in vocational maturity." The indicator' 
has now I^ecome a definer. The network of lines from specific ^ 
^measures to construct;^ has been short-circuited. 

• So the' question what to measure leaves us in a dilemma. 
On the one hand we don't want to swamp our evaluative enterprise 
jtfith meaningless rhetoric about goals that gives us no clue to 
measurement of progress. On the other hand, we don't want to 
IjLmit our observations to trivial and low-level behaviors that 
are directly coachable under such conditions as performance con- 
tracting. 

So where, we must ask, is the middle ground between what 
Tumin calls "trivial precision and apparently rich ambiguity?" 
Let's see whetlier we can find* it in 'any of the criteria that 
have been kicking around for some years. 

First, we must face the problem of long-range vs. short- 
range criteria. Unfortunately, this has been a very slippery 
pjroblem. Like* a fussy fisherman who cannot eat what he can catch 
anc? cannot catch what he could ea^, the would-be evaluator has 
found angling for data- on long-range outcomes overtaxes his' 
patience and resources, while the short-term data that are easil.^ 
netted often lack nourishment or flavor and may well be thrown 
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back. The ultimate criteria for judging effectiveness of a 
full-scale vocational guidance program have been elusive. What 
many want to know is: Does guidance make a difference in 
people's careers? What kind of occupational success , adjust- 
mentr and satisfaction do they achieve? What contributions do 
they make to society? To fish for answers to such questions 
takes timer money, and control of many variables, 

Preciods few have even tried to conduct longitudinal evalua- 
tion studies ranging over a period of years, Rothney's follow-up 
of experimental and control groups beyond high school is a nota- 
ble exception. He used many criteria , such as amount of post- 
secondary education r achievement in college ^ promotions in jobs, 
satisfaction with ^current status and with intervening decisions 
and actions, (In general ^ differences between the experimental 
and control groups were small and not significant. But even if 
there had been significant differences ^ would the time-lag and 
changing conditions permit assurance that the same treatment 
would have equally favorable outcome today?) At any rate^ most 
evaluators of guidance ^ like those who evaluate other areas of 
the school curriculum^ settle for the kind of criteria th^y can 
net more readily. A comprehensive search of their creels over 
the last 35 years discloses, most cOTunonly, such criteria as 
student satisfaction with counseling; persistence in school; com- 
parisons of students' self-ratings witjfi test scores; judges' 
ratings of "realism" or "appropriateness" of "preferred occupa- . 
tions" named by students; the proportiofi Cf- a class expressing 
an occupational goal; the -constancy of expressed occupational 
choice over a. period of time (say, from nfnth ,to twelfth grade); 
the relationship between proportion of a high school class ex- 
pressing preference for each occupation and the latest census 
count showing proportion of working force in each occupation in 
the community; expressions of counselee satisfaction; improve- 
ment in counselees' school marks^; etc. (Incidentally , guidance 
has rarely made a significant difference in these variables. 
There is no clear reason why it should,) 

Notwithstanding consistent negative results, *these criteria 
may have had sbme utility for the objectives of guidance that 
were widely accepted up to about 1950, The increasing acceptance 
of recent development in guidance theory, however, has made the 
digestibility of^Suqh criteria increasingly dubious. Today, such 
data seem hardly .worth pulling from the stream; the would*-be 
evaluator must find other fish to, fry. It is evident that the 
- construct represented by all these long-range and short-range 
criterion variables was whether students had learned to make wise 
decisions. That is, were the outcomes better for the experimental 
group^than for the control group? 

But to evaluate the long-term' outcomes of decisions is not 
only difficult: It is presumptiuous . Tenriyson wrote, "no man 
can^be more wise than destiny," I would feel more comfortable if 
we changed the criterion from "Making Wise Decisions" to "Making 
Decisions Wisely." This shiftr the emphasis from content to pro- 
""cess, "Wise decisions" implies a understanding of outcomes and 



a mastfery over events to which we cannot aspire'jf "M^ing 

decisions wisely," on the. other hand, implies arn understanding of 

self and a ma^stery over processes which may be more attainable. 

It is in this sense of wisdom that Tennyson is contradicted by 

the old Latin motto, Fa to prudentia major (Wi'^dom is stronger 

than fate) • . * ^ 

> ** 

Suppose you were counseling students in the late 50 's or 
early 60' s, and heeded tHfe goal supported by Congress, under 
NDEA, to identify able students and encourage them to continue 
with their education and prepare for' certain high-level occupa- 
'tions. Of course, NDEA owed its existence partly to the shock 
of sputnik — so you might feel particularly effective, if with your 
guidance, one of your i^rightest and ablest students decided to 
become an '^aerospace engineer. How gratifyin^_for you to have 
done your duty by Congress and your prof essionrxgyl? now your 
former student is unemployed. Was his decision a wise one? 
Was Vour guidance good? . * 

The problem in identifying wise decisions, however, is not 
just th^ time-lag between the choice-point and the judgment day — 
the day when all the evidence on consequences of the choice is in. 
Nor is it just a matter of insufficient predictive validity. 
Predictive data are really historical d%ta, and our predictions 
are manifestations of what we have learned from history. Thus, if 
our ^predictors had perfect validity, we could extend the aphor- 
•"ism "Those who do not learn from history We condemned to repeat 
it," by addinq "and those who do learn from history are also 
condemned to repeat it." But in fqict we don't repeat history, 
even when events inaterialize as we /nave predicted. For there is 
always a surplus of e^^nts — there are more events than predic-*. 
tions. The outcomes of decisions exceed the purposes of decision- 
makers.. P^Y decision that is not trivial has ramifications with- 
out end. Each outcome then may generate new purposes and pre- 
dictions may be buried under this landslide of outcomes and 
decisions arid outcomes. * , * 

Consider^ as a somewhat painful example, the decision of 
the U.S. Government to intervene in Vieiyiam. One could argue, 
and indeed the government has argued, that this was a wise de- 
cision in that the purposes of this decision were (and are being) 
fulfilled as jpredicted. But surely the government does not main- 
tain that all the outcomes of-' that decision were predicted, and 
it has built no granaries , for storing the surplus events until 
such time as we need them — or at least are better able to cope 
with them. As the Pentagon papers haVe made clear, thfe. fault in 
the decision to intervene in Vietnam was in the process, not just 
in the outcome. Suppose the outcome had been somewhat different: 
suppose we had had a great military success there — had "brought 
the coonskin home and nailed it to the wall." Would that mili- 
tary success have wiped the slate clean of the flaws in decision- 
making? Would it have justified pur decision? Perhaps it would 
have prevented the moral questi6n3 from being raised — as when ^we 
interve'ned in the Dominican Republic— although it is unlikely 
that We could have "yon" in Vietnam that fast, or with less 
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publicity and condemnation than-^he Russian interventions in 
Hungary and Czechoslov!akia . At least a few voices — voices like 
Jim Alien's — would have cried out in the wilderness about the 
moral issues. But a victorious outcome would have prevented 
widespread popular concern. "My Lai ^ and tiger cages and one-marf 
election races would never have plagued us, and the wholp inci- 
dent^ would have soon blown over in the; media and the publ;ic • 
consciousness. Would that military succesa have madd the decision^ 
a wise one? Would the decision have^been made any more wisely? , 

For the sake of argument, let us suppose that we have pre- 
dicted and can evaluate the ramified outcomes of this decision to 
intervene as in some sense superior to those which would have 
been pr^oduced hy any alternative decision. Even then, what would 
the substantive 'payoff of this decision reinforce? The content of 
this decision itself? But this same decision is not likely to come 
up again. We only pass this way once. Then we would be hard put 
to claim an increment in wisdom from the coijteht of this decision. 
fPhe content of a single wise decision is not likely to be trans- 
ferable to the next decision, and the next.. 

In fact, what one learns from the multitude of* real-life 
outcomes may or may ^ot be relevant to wisdom. Like Mark Twain's 
cat, who learned from sitting on a hot stove never to sit on any' 
stove again, we may learn from these outcomes m\>re "wisdom" than 
is i^ them. For example, the current overflow of outcomes from 
the Vietnam decision might teach us to revert to isolationism 
(in contradiction to the "lessons" from previous decisions and 
outcomes).. The little boy who is spanked for turning the faucets' 
on full blast and flooding the bathroom may learn not to wash 
his hands and face. 

It is these tendencies tcT^ "generalize" that lead the behav- 
iorists to concern themselves with what Skinner calls "contingen- 
cies" in their schedules of reinforcement. Or as 0. H. Mowrer 
once put it, tin a classroom discussion ot omj^pt his learning, 
experiments, "You've got to be smarter than The ^ rat . " Well said, 
since such an approach to defining wisdom in tems of outcomes 
requires that wisdom reside in the experimenter — or counselor, 
not in the subject — or student. But this is where the presump- \ 
tion comes in: do we as counselors know which decisions are wise? 

Here one may object, are there not "universally desired" 
outcomes that represent a cultural, consensus or folk wisdom fflir 
which the counselor may serve as spokesman? Let us grant this- 
^while noting that we may retain some squeamishness about our 
ability to identify such universals even in retrospect, let 
alone in advance. Presumably, we can teach jstudents to make 
these decisions that lead — with a high degree of probability and 
with low risk — to universally desired outcomes. 

But when we have identified such universale and induced 
students to learn them, we are riot really concerned with decision- 
making — or with guidance. Then we are concerned with indoc- * 
trination. A large part of an individual's schooling consists of 
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such indoctrination. T^e distinctive concern of guidance, 
however, is not with the universal s, but with the "alternatives" — 
toward which the culture tends to be more permissive. 

However, I must express some dissatisfaction with the term 
"alternatives." The individual is not always constrained to 
choose from clearly shaped ^alternatives that are already "there" 
like the options in a multiple-choice te,st. He often has some 
opportunity to construct, or create, his own options — in the 
sense that the poet creates his verses ^ perhaps ^creates alterna- ♦ 
tive verses, before choosing the ones he wants. He is not Hierely. 
choosing alternatives from his total vocabulary, any more than 
the painter is, merely choosing colors and lines from an existing 
pool of options. He does npt find his new and unique combinations. 
Variations, Sfid trarisformations by considering all possible per- 
mutations. Fifty chimpanzees typing for *f if ty 'years might compose 
the complete works of Shakespeare^ but they wouldn't know how to' 
write a new work of similar quality. In terms of content and 
outcomes, they might have made "wise" decisions y and y^t they 
wou Id be none the wiser . Rs critics, we can evaluate the poet's 
decisions, recognize t)iem as creative, or wise,, and teach someone 
^ to memorize them. JWe can^ even 'derive and appTy rules for trans- 
fer of content. For example, we can analyze a line like "Now is 
the winter of our discontent" and recognize an association be- 
tween emotion or state of mind and a seaso'n of the year, in which 
season is used to represent feeling. No doubt, a computer could 
be programmed to ring the changes, on this kind of association^ 
with such results as "Now is the summer of my happiness/' Now is 
the spring of my joy/' Now is the autxamn of my melancholy," etc. 
ad nauseam . But could it ever make the long leap from this last j 
to reach "My way of life is fallen into the sere, the yellow 
leaf . . ."? This illustrates, I think, the gap between recog- 
nition of a creative r or wise^ decision and the ability to make 
one. How often the best and wisest decision i^ riot to choose 
between historically "given" alternatives, but- to construct a , 
new option. Like able students who squirm at being forced to 
choose the best of five bad options on a multiple-choice test 
question, our wisest decision-makers can sometimes think of a • 
better response. than any given. 

I hope that all this suggests an "alternative" to defining 
wisdom in terms of outcomes. How a choice comes out, and even 
how one chooses between alternatives, may be less imp6rtant than 
how one constructs alternatives. In this view, wisdom derives 
not from the outcome of a decision but from the process of de- 
cision-making. And our greatest folk-wisdcm, our mast compel- 
ling "universal," may apply most directly to the process of con- 
structing and choosing alternatives. 

For may we not regard democracy itself as an evolving pro- 
cess of decision-making? It is its processes, not the content of 
any one^olicy decision, that make it distinctive. 

We recognize, as a crucial characteristic, for the processes 
by which we ideally make national policy decisions, that our 
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society, is pluralistic. On every issue competing interests and 
pressure groups are heard. Sometimes they differ on predictions 
X)f outcomes — for excimple, the effects of a tax increase on the 
economy. More often, and more significantly, they have different 
definitions of desirability, different objectives, even when they 
agree on predictions Qf. outcomes. How do these differences get 
resolved ^wisely"? The necessary condition, we believe', is 
freedom^-the open marketplace of ideas, in which every voice can 
be heard and judged. Out of this confrontation of competing 
values, the legislative or executive can find — or claim to fiAd — 
a consensus for decisipn, to be translated into a mandate for 
action. But it does not stop there. The process is ongoing, 
permitting revision of content .in accordance not just with out- 
comes, but also with changes in values and objectives. This - 
provision for change, this ability to accoidmodate to new sit- 
uations and circumstances, has perhaps insured the ,3urvival of 
democracy, up till now, «>through many vicissitudes. (Our ^ability 
to reverse our decision on Vietnam is' a sign of strength, not 
of weakness.) 

Need I belabor the analogy with individual cjecisiota-mal^iing? 
The individual, too, recognizes that he must choose between 
competing values. How then does he make 'order out of the' 
rabble of impulses that beset him? They should be neither 
suppressed nor blindly obeyed, but brought under the rule of 
reason, each given* "equal time" and attention. The individual, 
like the nectlon, must hold himself open and receptive .to dif- 
ferent values., allowing each to speak to him as loudly as the 
others. This profijess involves active and systematic examination 
and exploration of competing values. ^ 

One way in which he can examine values is to study their 
sources. Here we see a nice articulation of education and guid*- 
ance. If a major purpose of education is t6 transmit the culture, 
an important purpose of guidance is td' help the individual come 
to terms with the culture — that is, the choices he makes will in- 
dicate how he sees himself in the culture. But first he must see 
the culture in himself. So his first question must be, where 
have my values come from? Then he will be bettei; prepared to ask, 
where are they taking me? 

When the student has taken full pognizance of the range of 
values in the culture, and has formulated his. own value system 
quite explicitly, he will be ready to- lay his values on the 
line in making a decision. The specifics of a strategy for ac- 
complishing this I have described elsewhere and will not have 
time to go into now. But I want to emphasize that with the in- 
dividual, as with the nertiion, deci s ion-gnaking should be an on- 
going process, subject to continual revision. Otherwise, he may 
run afoul of the warning that "the only thing worse than not 
getting what you want is getting it." 

In shunning a definition of wise decisions in terms of con- 
tent, or predicted outcome ^ I have assumed that experience does 
not teach us what will be bei^t for the individual (or society) 



cxcppt freedom to work things out\- Thus, I have defined the be^st 
choice a'S the choice that is most nearly free. But I do not 
define freedom as complete' laisser-faire. Rather, it is the 
freedom (expressed by Shaw in the preface to Man and Superman 
and quoted by Freud in contrasting his "reality principle" with 
his "pleasure principle") "to be ab.le to choose the line of 
greatest advantage .instead of -yielding in the path of least re-, 
sistance." So without directing the content of an individual'' s / 
choice, w€i do think we can help him in the process of choosing • 
This emphasis on ptocess does not pretend to insure the "right .. 
choice — except insofar as the right choice is defined as an in- 
formed and rational choice. Our bias — our conviction — is that 
in education enlightened processes are intrinsically important. 
^Therefore, we j^end our efforts to, increase the . student ' s under- 
standing of the factors involved in chpice (imperfect though our 
own understanding may be) so that he' can take responsibility 
for his oVm decision-making, examine himself and explore his 
options in a systematic and comprehensive w^y, '^take' purposeful 
action iji testing hypotheses about himself in various situations, 
and exerci^se f lejTibility in devising a^lternate plans. 

In short, we don't want to play' the decision-makincf game 
for him. We want to help him master the strategies fo^r rational 
behavior in the face of uncertainty (which may be the nearest 
he can get to wisdom) so that he can play the game effectively* 
himself. 

Horace, in one of his satires, asked "Who then is free?" 
and answered ."The wise man who can govern himself." 

So now at last we move on to the question of how evaluations 
can be made. In an interesting paper just published, Hartnett 
has pointeid out some of the weaknesses of the classic model of 
evaluation, which involves such elements as (1) behaviorally 
defined objectives, (2) the random assignment of subjects to 
treatments, (3) clearly differentiated treatments, and (4) cri- 
terion measures chosen or developed on the basis of the behavioral 
objectives. suggests that dissatisfactions with this model 

are leading to two important changes: a concern for the con-* 
sequences, not just the objectives, of a treatment, and a style 
of inquiry which is exploratory in nature rather than attempting 
to apply in life sitiuations the kinds of controls and irtanipula- 
tions that are feasible only in the scientific laboratory. 

Pace has typified the new models in this way: "The spirit 
of the evaluator Should be adventurous. If only ^hat which could 
be controlled or focused were evaluated, then a great many 
important educational and social developments would never be 
evaluated . . . that would be a pity." 

/' 

/In guidance, this exploratory set must be eitip'hasized. We 
have'' no neat evaluation packages all wrapped up and* ready to use. 
For example, a number oif people have developed what purport to - 
be measures of "Vocational Maturity." Can any of these measures 
be recorimended for use? 
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One of the best known measures is John Crit6s • VDI/ an 
inventory keyed to the responses of 12th-graders Extensive 
research has been done on this instrument — for example, on 
elimination of variance attributable to acquiescent response 
set. Yet Super has ..criticized the instrument on just these 
grounds: Vocational Maturity, as defined by VDI, ^fteans saying 
no. '(There we v see it again, the instrument take^n as the definer 
ratiier than the indicator of a construct.). Another criticism 
involves the use of 12th-graders • responses as the keyed re- 
sponses: a. group of 10 counselor educators and vocational 
psychologists disagreed with the keys for a number of items. 

Back in the 1950! s, I developed an objective test that I 
am not particularly proud of. It attempted to find out whether 
students had -mastered certain concep^ts involved in self-appraisal, 
getting and using information, and (^ecision-making . At the same 
timei afnd in connection with the same project, we commissioned 
Warren* Glribbons to develop an int^view. schedule , known as 
Readiness for Vocational Plarining, to see whether students were 
actually applying those concepts to their own educational and 
occupational decisions'. We were evaluating a work test for 
group guidance, and found highly /isignif icant differences between 
experimental and control groups — for example ,. experimental stu- 
dehts scored velry significantly higher on the test and also 
showed very significantly great^ awareness of. their own values, 
better ability to define their Values and to describe t;he role 
their Values jlay in their decision-making, arid so on. A group 
of professionals in guidance, listening to tapes of^ the inter- 
views without knowledge of th0 scales or scores, ranked the stu- 
dents in the same order on ••vocational matufity^^- as the total 
scores did.. Grib^ons & Lohneis have now converted the interview 
schedule into a questionnaire form, RCP. 

Super and his colleagues have recently developed a Career 
Questionnaire that also purports to jneasure vocatio;ial maturity. 
It includes scales called Concern with Choice, Acceptance of 
Responsibility, Occupational Information, Work Experiences, 
Crystallization of Interests, and so 6n — rubrics derived from 
the Career Pattern Study. 

Westbrook has been developing a series of Vocational Maturity 
Tests, including some of the items from my ol^ test. The items 
tap various kinds of inf ormationr. Course and curriculum* Selection, 
Planning, Goal Selection, etc. 

These are -the major standarized efforts I know of to get at 
the construct, vocational maturity, and they are all well con- 
ceived; they are good tries. I am not damning them with faint 
praise. I just want to forewarn you that you may be disappointed 
when you see the actual instruments and study them item by item. 
You will agree, I am sure, that even though they may be indicators 
of vocational maturity, they are not definer s of, it. 

The questions getting at facts about specific occupations 
hardly seem appropriate for students who may have had no interest 
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whatsoever in those occupations then, t<x), a number of 'the 
items depend on occujjation^A preferences^ expressed by the stu- 
dents — for i^ample, Super is concerned with Wisdom of the Vo- 
cational PiJ'ef erence" and with "Consistency of Preference." 

The title of ^n occupation, however, is probably a poor 
indicator of what choosing an occupation means to an individua^i,. 
More relevant questions might be, in his view, how important an 
element of his life is represented by occupation? What kinds 
and amounts of satisfaction does he hope to derive from it? 
What differentiations does he discern between .occupations in 
capability of providing such satisfactions? How much control? 
over his choice and responsibility for his choice does he appear 
to -exercise? What role do predictive da%a play in his choosing? — 
does he consider them? Is he dominated by them? What risks is 
he willing to take to achieve the occupational satisfactions he 
says he wants? What decision rules does he employ? What re- 
sources does he use? What reality tests of his perceptions and 
predictions has he made, or does he plan to make? How has he 
coped — how will he cope — with obstacJ.es and dif f iculties?"^'Tl^s 
hQ, formulated viable alternative plans? How explicit and' con- 
sistent is his reasoning about these questions? 

^ Once we have probed beneath the surface of choice to get at 

such underlying perceptions, attitudes, an<f^ rationales, we may 
find ourselves with much richer criteria of growth and vocational 
development. Dr. Bingham's efforts to get at the dimensions 

r along which individuals consti;ue occupati^bns — using an adaptation 
of Kelly's Role Concept Repertory Test — is a step in this direc- 
,tion. Some of my associates and I have developed and used in an 
exploratory way, interview schedules to qsJi at students' occupa- 
tional constructs.. We had them look at occupai:ions and -sort them 
in the way that Kelly's Role Concept Repertory Test would. Vie 
asked them to do some analysis of their own values. We found 
that son^(6 of the more prpductiver. questions were things like this: 
"Now sit Hack and turn your imagination loose. Try to describe, 

♦as fully you can, what you would regard as an ideal of 'dream' 
occupations. It can be a real occupation, or one you invent." 
And then aa the student began to fill out the descifiption of 
this occupation we asked questions which pushed him toward fil- 

' ling it out more and more. Then after he had done all this, we 
said, "In view of what you said about an ideal occupation, why 

didn't you decide to become a ?" Here we had to be 

fast on our feet and think of an occupation that seemed to fit 
his ideal, instead of whatever occupation he had named as his 
pr^feired choice. This is very interesting because it puts him 
in the position of trying to work his way through his reasons 

"^or choice. Here he's given a choice and described something 
which may fit a little bit better some* other choice. As he works 
his way out of this, as he tries to explain and rationalize what 
he has done, he's really challenging ^limself to pin down his 
valuies, to pin down more explicitly what it is he wants, And then 
we asked, "Now reverse your field and think of the worst occupation 
you can. If the other was a 'dream' this would be a 'nightmare': 
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Describe it.". Again, we locked to see what values were involved • * 
And again we saw whether these were contrasted with' or agreed in' 
some respects with the occupation of his expressed preference. 
Of course the interview itself had its effects. One .probably 
cannot measure the status of an individual's decision-making , 
without influencing it. For instance, at the end of- interviews 
with junior college .students we got comments like this: the 
interview "extended my ideas about what to look for in an occupa- 
tion," * -"made me think about why I was making my choice," and so on. 
For example, it seamed to have a particularly strong impact on 
ones student who had appeared especially firm and specific in hi^ 
plan to become a chemical engineer. Working as a draftsman after' 
his graduation from high school (where he said he had been 
"pushed ihto" a vocational curriculiun by his guidance counselor) , 
he had had a particularly good opportunity to observe 'chemical 
engineers at work and had an linusually tho^ugh knowledge of 
their work activities. His perceptions (ill? the comparisons of 
occupations) seemed fixed almost exclusively on one construct: 
whether- an occupation offer^d an outlet for' scientific interest 
and inventiveness, or. not. T]ie sole deviation involved a dis- 
crimination between occupations in' terms of altruism — oppor- 
tunity to help others. The systematic exploration and examina- ^ 
tion that accompanied his sealing of values brought out more 
explicit recognition bf Altruism as a value of §ome importance - 
to him*. With this* discovery , other, vaiues of which he had i:iot 
been fully aware also came into focus' as jgui^te important to hinj: 
notably Variety and Autonomy. At the end he said that ^the in- 
terview had "brought to the surf ace* values I've held liut never 
recognized. That shate^s me. ... If I had two ^••Ves to lead, 
for one of them I'd go into the Peace ^ Corps as 'soOi">as I finished, 
collecfe^ Maybe then I>*d try to become a high school teacher or; 
counselor, or a community worker. But I cam^ up the hard way.* , 
There are things I s^e now I want to d©, but I can't do them un- 
til I get firm ground under me. 'I'm still detemiine^d to become 
a chemical engineer. Not .like a machine, though, but like a 
person^ " ^ * \ 

If you ca.ji't measupeya Condition without changing it, does 
that mean you should not try to measures it? No, not 'even if it , 
is a differential influence, affecting different students in 
different ways^ After all, people encounter many common e^" 
periences that have differential effects, and this attempt at 
measurement is only one of such an unknown number. The dif- 
ferential effect may indeed be part of the substance of what we' 
are trying to investigate. Messick has pointed out that tra- 
ditional questions in education and psychology have frequently 
spawned answers that are either downright wrong, in that they 
summarize findings "on the average" in situations ,where a hypo- 
thetical "average person" dimply doesn't exist, or elfee are 
Seriously lacking in generality, in that they fail, to take ac- 
count of the multiplicity of human differences and their inter- 
actions with envi;ronmental circumstances. 

An example is the "horse iTace" question typical of much 
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educational research of past decades: Is treatment A better- 
than treatment >B?* Such questions dre usually resolved by com- 
paring average gains inf achievement for student's receiving 
treatment A with average^ gains for students recdiviag treatment 
B. But suppose treatment, A is better for certain kinds of stu- 
dents and treatment B better* for ot>\er kinds of^ students? A 
' completely different evaluation of ^tiie. treatments might result 
* if some other, more complicated questions ?.ad been asked such 
^'s "DOythese treatment^ interact with <aifferences in personality 
^d cognitive characteristics of students — or with differences in 
thfeir educational hist03?y, or family background, or community, 
or culture--to produce d'if f ^irential effects upon achievement?" 

Hard upon this warning of the complexity o^ evaluation in 
guidance, let me quote again from Henry Dyer: . . _ 

"The .term educational accountability, -as' used 
most recently by certain economists, systems analysts,^" 
and the like, has frequently been based on a concep-- 

' tualization tt^at tends, by analogy, to equate the 
educational process with the type of engineering 
process that applied ;to industrial production, • • . It 
must be constantly kept in mind that the educational ^ 
process is not all foufs with an industrial process; ' 
it is a social process in which human beings are 
caijtinually divceracting with other human beings in 

. ways that are imperfectly, measurable 'or predictable. 

Education does not deal with inert , raw/ materials , but with 
living minds that are instinctively, concerned first 
with preserving their own inte^ity and second Vith 
reaching a meaningful accommodation with the world 
around them. The output of ttie*educationai process ^is i " / 
never a- "finished product" whose characteristics can 
be ' rigorously specified in advance; xt is an individual 
who is sufficiently aware of his own incompleteness to 
make him want to *keep on growing and learning and ^ 
trying to solve the riddle of his own existence in ^ 
a world that neither he nor anyone else can fully 
understand or predict." 

Despite these problems, evaluate we must. And so I come - 
back, in conclusion, to my .third reason for why we evaluate. 

My third reason for evaluation, despite all^ its snarls and 
pitfalls, is simply this. If we believe in trying to heip stu- 
dents make career decisions wilsely — that"' is, make rational and 
informed decisions — tKfeflNwe^ust also," in all honesty, believe 
that guidance practitioners should make their professional 
*^ decisions wisely. We have to provide students wit!h a model for 
decipion-making behavior — and that is just wh^t an evaluation 
^process is. It is a commitment to use of information and rea-"*^ 
^^n, to rational behavior unde^: conditions of uncertainty. 
So.^-like the students — we must take resppnsibility for eval- 
uation. We must make our professional values explicit, exSunine 
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and explore them. We mu^t for^nulate hypotheses about the 
effects of our activities, and try to get feedfeack. We must 
revise our iiypotheses, plans, and activities in the light of 
new information. 

When we evaluate, we commit ourselves to a continuous 
process of. dec4.sion-making. It is a commitment we should wel- 
come. The- methods and the product may. leave much to be desired 
But let lis irealize that commitment to the process itself may 
)De\ a powerful indicator of how good a school guidance { prograun 
is. . 



